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1.0  INTRODUCTION 


This  is  a  final  report  on  the  development  of  the  Multiprocessor 
Shared  Memory  (MSM) ,  a  contractual  and  in-house  effort  by  AFWAL/AAAF  for 
the  Avionics  System  Analysis  and  Integration  Laboratory  (AVSAIL)  located 
at  the  Avionics  Laboratory.  This  report  outlines  the  developmental 
aspects  of  the  MSM  and  provides  a  general  technical  description  of  its 
design.  A  detailed  technical  description  is  found  in  the  "Critical  Item 
Product  Function  Specification  for  a  Multiprocessor  Shared  Memory 
(MSM)", AFWAL-TR-83-1024,  Vol  I.  Included  in  this  report  is  a  description  of  thi 
MSM  prototype  currently  in  use  at  AVSAIL  and  its  specifications. 

2.0  BACKGROUND 

2.1  AVSAIL /Rea 1-Time 


When  AVSAIL  began  operation  in  1975,  the  DECsystem-10  mainframe 
computer,  eight  PDP-11  minicomputers,  and  the  DEC  DMA10  were  designated 
as  the  primary  hardware  components  dedicated  to  support  the  real-time 
efforts  of  the  System  Avionics  Division.  The  DECsystem-10  was  to  be 
used  basically  for  the  computational  loads  required  of  simulations.  The 
satellite  PDP-11' s  were  to  provide  assistance  for  the  controls  and 
displays  of  the  cockpit  simulators  and  drive  the  graphics  system  for 
"out-the-window"  scenes.  The  DMA10  (direct  memory  access  device)  was  to 
transfer  data  between  the  mainframe  and  satellite  computers. 

However,  within  a  couple  years,  requirements  were  such  that  AVSAIL 
could  not  adequately  support  two,  large  scale  projects  of  the  System 
Avionics  Division.  The  real-time  cockpit  efforts  of  the  Digital 
Avionics  Information  System  (DAIS)  and  the  F-16  Independent  Assessment 
Simulator  (IAS)  could  not  run  their  simulations  simultaneously.  The 
F-16  IAS  simulation  program  crashed  frequently  when  running  concurrently 
with  DMAX,  a  program  used  for  file  transfers  between  the  PDP-11  and 
DECsystera-10.  These  problems  were  found  to  be  the  result  of  the  DMA10, 
the  link  between  the  DECsystem-10  and  PDP-11  minicomputers.  Figure  1 
illustrates  the  problem  that  occurred  during  the  F-16  IAS  simulation. 

2.2  DMA 10 


The  DMA10,  produced  by  the  Digital  Equipment  Corporation  (DEC),  is 
a  programmable  word  transfer  link  which  connects  the  DECsystem-10  to  as 
many  as  eight  PDP-11  minicomputers.  It  uses  a  circular  scanner  as  a 
means  of  processing  memory  requests.  The  DMA10  uses  the  lower  256K  of 
DECsystem-10  memory  as  the  location  where  the  PDP-11  can  access  during 
read  and  write  operations.  This  common  "window"  can  be  as  large  as  24K 
for  the  PDP-11.  Communications  between  processors  is  done  via 
interrupts . 

The  main  limitation  of  the  DMA10  has  been  its  speed.  Figure  2 
illustrates  the  transfer  time  capability  of  the  DMA10.  Data  transfers 
were  not  fast  enough  for  the  Fire  Control  Computer  (FCC)  on  the 


FIGURE  1.  AVSAIL  REAL-TIME  CONFIGURATION  FOR  F-16  IAS 


TDATI  =  TRANSFER  TIME  FROM  PDP-11  TO  DECSYSTEM-IO  MEMORY  (PDP-11  READ) 
(excludes  ANY  UNIBUS  CABLE  DELAYS  ANr  latency). 


MIL-STD-1553  Multiplex  Data  Bus,  when  three  or  more  DMA10  channels  were 
active  during  an  F-16  IAS  simulation.  Another  limitation  is  the 
requirement  that  all  programs  using  the  device  must  be  locked  within  the 
first  256K  of  core  memory.  The  DMA10  is  limited  to  18-bit 
addressibility  of  the  DECsystem-10  memory. 

2. J  MPA 


The  first  attempt  to  replace  the  DMA10  resulted  in  the  preparation 
of  a  development  specification  for  a  Memory  Priority  Access  (MPA) 
device.  The  MPA  proposed  a  priority  encoder  scheme  which  was  expected 
to  greatly  enhance  the  throughput  capability  over  the  circular  scanner 
method.  However,  after  examining  the  detailed  timing  of  the  system,  it 
was  concluded  that  PDP-11  access  to  DECsystem-10  memory,  via  the  MPA, 
was  not  faster  than  the  DMA10.  Another  drawback  of  the  MPA  was  the 
functional  partitioning.  Since  PDP-11  channel  logic  was  quite  complex, 
additional  PDP-11  units  would  significantly  increase  the  total  cost  of  a 
system.  These  drawbacks  of  the  MPA  led  to  a  new  concept,  the  shared 
memory  system,  for  a  faster  transfer  of  data  between  processors. 

2.4  MSM 


The  MSM  provides  common  memory  for  the  DECsystem-10  processor  and 
PDP-11,  and  interprocessor  communication  control.  It  can  link  eight 
PDP-ll's  to  a  DECsystem-10  as  does  the  DMA10.  Its  main  advantages  over 
the  DMA10  are  as  follows:  shared  semiconductor  memory  for  faster  PDP-11 
access,  22-bit  addressibility  for  up  to  four  million  word  addressing,  up 
to  124K  of  "window"  space  for  the  PDP-11,  more  flexible  DECsystem-10 
processor  configurations,  and  remote  PDP-11  operations  of  up  to  150  ft 
with  electrical  isolation.  Figure  3  illustrates  the  transfer  time 
capability  of  the  MSM.  The  MSM  easily  allows  for  two  simultaneous 
simulations  to  be  made  using  the  MIL-STD-1553  Multiplex  Data  Bus  without 
transfer  time  restrictions.  Block  diagrams  in  Figures  4-6  show  the  MSM 
and  the  break  down  of  its  major  subsystems. 

3.0  MSM  Development  (Contractual) 

An  effort  to  replace  the  DMA10  was  directed  by  the  AAAF  branch  with 
support  from  Simulation  Technology,  Inc.  (SIMUTECH)  under  a  Time  & 
Materials  (T&M)  contract.  The  "Specification  for  the  DEC-10  Memory 
Window  Control  System  (MPA)"  prepared  Sep  78  established  the 
performance,  design,  development,  and  testing  requirements  of  the  new 
MPA  device.  In  May  79,  an  MPA  Version  2  (MSM)  concept  was  presented, 
giving  a  much  improved  design  over  the  original  MPA.  Within  the 
following  month,  a  "Specification  for  the  DEC-10  Multiprocessor  Shared 
Memory  (MSM)"  rough  draft  was  initiated  along  with  the  start  of  the 
detailed  subsystem  design. 

The  MSM  design  requirements  were  based  on  inputs  from  both  AVSAIL 
and  the  Integration  Facility  for  Avionics  System  Testing  (IFAST)  located 
at  Edwards  AFB.  The  MSM  was  designed  to  interface  with  any  PDP-11 
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Figure  4.  Multiprocessor  Shared  Memory  (MSM)  Block  Diagram 
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Interprocessor  Communications  Subsystem  <ICS)  Block  Diagram 


minicomputer  (UNIBUS  device)  and  the  KUO  or  KL10  based  DECsystem-10 
mainframe  computer.  Other  design  requirements  which  were  determined  for 
the  MSM  are  included  in  Section  6.0. 

In  Aug  79,  a  prototype  was  planned  for  fabrication  and  testing. 

Its  purpose  was  to  verify  the  functional  concept  of  the  MSM  using  one 
KI10  processor  and  one  PDP-11  minicomputer.  It  was  to  be  delivered  by 
SIMUTECH  complete  with  documentation  by  the  end  of  Jun  80.  In  Feb  80, 
an  MSM  critical  design  review  was  held  for  AVSAIL  and  IFAST  personnel. 
This  presentation  gave  an  overview  of  the  MSM  design  and  its  advantages 
over  the  DMA10  system  currently  in  use.  A  slip  in  schedule  later 
resulted  in  only  a  two  week  period  for  system  debug,  once  the  MSM  was 
brought  to  AVSAIL  and  connected  to  the  DECsystem-10.  At  the  close  of 
the  contract  (30  Jun  80) ,  SIMUTECH  delivered  the  prototype  with  only  30% 
of  the  hardware  debugged  and  50%  of  the  software  diagnostics  written. 

The  contract  was  not  renewed  with  SIMUTECH  and  therefore  an  in-house 
work  unit  was  initiated  to  continue  the  development. 

4.0  MSM  Development  (In-House) 

The  initial  approach  of  this  work  unit  (Jul  80)  consisted  of  the 
following:  completing  the  MSM  feasibility  prototype  and  verifying  the 

functional  concept,  integrating  the  MSM  prototype  into  AVSAIL  by 
providing  a  ninth  PDP-11  channel  for  the  DECsystem-10,  and  contracting 
out  for  two  production  units  for  AVSAIL  and  IFAST.  The  MSM  production 
unit  for  AVSAIL  was  to  replace  or  coincide  with  the  DMA10. 

Hardware  debug  began  in  Sep  80,  concentrating  initially  on  the 
PDP-11  side  of  the  MSM.  Once  PDP-11  diagnostics  ran  successfully  on  the 
prototype,  debug  began  on  the  DECsystem-10  side.  By  Dec  80,  both  the 
PDP-11  and  KI10  processors  were  able  to  separately  access  MSM  memory. 

The  next  three  months  involved  software  diagnostics  development  and 
checkout,  and  continued  hardware  debug  with  subsystem  modifications.  A 
prototype  demonstration  was  given  to  the  AAAF  in  Mar  81  to  verify  the 
functional  concept  of  the  MSM.  This  marked  completion  of  the  MSM 
software  diagnostics. 

Due  to  changes  in  computer  requirements  for  IFAST,  the  requirement 
for  MSM  production  units  was  dropped.  IFAST  chose  the  VAX  11/780, 
instead  of  the  DECsystem-10,  for  real-time  processing.  AVSAIL  continued 
development  of  the  prototype  and  planned  to  expand  it  from  one  to  two 
channels  to  support  a  general  purpose  cockpit  development. 

To  further  verify  the  functional  concept  and  to  demonstrate  its 
real-time  capability,  plans  were  made  to  run  the  F-16  IAS  simulation 
program  using  the  MSM.  The  MSM  was  to  replace  the  DMA10  channel  used  to 
link  the  models  running  on  the  DECsystem-10  with  the  controls  and 
displays  of  the  PDP-11.  Modifications  to  the  original  F-16  IAS  software 
were  required  including  the. F-16  Executive  and  the  PDP-11  Controls  & 
Monitoring  software.  The  TOPS-10  monitor  and  SIMSUB  (Simulation 
Subroutine)  on  the  DECsystem-10  also  required  modification. 


During  checkout  of  the  modified  F-16  IAS  software  in  Jul  81,  a 
problem  occured  with  the  DELCO  362F  Fire  Control  Computer,  the  prime 
controller  in  the  simulation.  The  FCC  acquired  parity  errors  during  the 
software  loading  of  the  Operational  Flight  Program  (OFP).  The  problem 
was  in  the  hardware  and  the  unit  had  to  be  shipped  to  DELCO  for  repairs. 
This  caused  a  five  month  slip  in  the  testing  schedule.  By  Jan  82,  the 
computer  was  repaired  and  software  testing  was  resumed. 

Two  F-16  Executive  versions  were  written  for  the  MSM.  The  first 
version  considers  the  MSM  to  be  the  sole  channel  in  the  simulation. 
Real-time  data  can  be  monitored  by  a  VT-11  terminal  connected  to  the 
PDP-11.  The  second  version  requires  use  of  both  the  MSM  channel  and  a 
DMA10  channel.  The  DMA10  channel,  as  in  the  original  F-16  IAS 
demonstration,  is  used  for  the  "out-the-window"  graphics  display.  The 
second  version  of  the  F-16  Executive  was  designated  for  the 
demonstrations  of  MSM.  These  software  mods  were  completed  in  Apr  82. 


During  the  software  testing,  further  hardware  bugs  had  surfaced 
which  had  not  been  detected  earlier.  Modifications  to  the  Memory  Access 
Sequence  (MAS)  and  the  DEC-10  Memory  Bus  Interface  (MBI)  subsystems  were 
|  necessary  to  correct  problems  with  the  read-modify-write  circuitry.  In 

May  82,  the  FCC  developed  hardware  problems  again  which  posponed  a 
demonstration  for  the  System  Avionics  Division  (AAA).  This  initiated  a 
software  mod  that  bypassed  the  FCC  in  the  simulation.  A  demo  of  the 
F-16  IAS  simulation,  using  MSM  without  the  FCC,  was  given  to  AAA  in  Jul 
82.  This  marked  conclusion  of  the  hardware  debug  of  the  MSM  prototype. 

The  following  month,  AAA  cancelled  the  proposed  Dual  Channel  MSM. 
This  prototype  enhancement  was  to  provide  an  additional  PDP-11  channel 
and  a  second  DEC-10  memory  bus  interface.  The  decision  to  stop  further 
MSM  development  was  based  on  the  purchase  of  new  and  more  powerful 
computers  to  handle  the  real-time  simulations  of  AVSAIL.  These  new 
Harris  computers  as  of  Oct  82  are  to  take  over  the  real-time  loads  of 
the  DECsystem-10  in  late  FY83. 

5.0  Description  Summary 

The  MSM  is  a  shared  memory  and  interprocessor  communications  and 
control  device.  It  is  exclusively  designed  as  an  interface  between  the 
DECsystem-10  and  up  to  eight  UNIBUS  devices,  such  as  the  family  of 
PDP-ll's.  Tt  can  configure  up  to  four  KI10  or  KI.10  processors  of  the 
DECsystem-10.  The  MSM  sequentially  scans  twelve  ports  without  regard  to 
port  priority.  It  provides  read  data  to  any  port  within  200  ns  of 
request  recognition.  The  MSM  contains  control  and  status  registers  for 
each  PDP-11  and  DECsystem-10  processor.  With  these  registers,  up  to 
four  DECsystem-10  processors  can  configure  any  PDP-11  and  its  access  to 
memory.  The  MSM  provides  interrupt  capability  for  both  the  DECsystem-10 
processors  and  the  PDP-11  minicomputers.  The  MSM  can  configure  for 
four-way  interleaving  by  expanding  its  memory  into  four  sections  for  use 
by  a  maximum  of  four  KL10  processors. 


6.0  Design  Features 


Listed  below  are  the  features  incorporated  in  the  MSM  design: 

-  18-bit  addressibility  on  the  PDP-11  UNIBUS 

22-bit  addressibility  on  the  DECsystem-10  Memory  Bus 

-  Functional,  electrical,  and  physical  compatibility  with  the 
DECsystem-10  Memory  Bus,  DECsystem-10  I/O  Bus,  and  UNIBUS 

-  Up  to  eight  UNIBUS  interfaces 

Up  to  four  DECsystem-10  I/O  Bus  interfaces 

-  Up  to  sixteen  DECsystem-10  Memory  Bus  interfaces  for  four-way 
interleaving  (4  MSI  configuration) 

-  Up  to  128K  of  DEC-10  words  of  internal  semiconductor  memory 
which  is  expandable  in  16K  block  increments 
Software-selectable  mapping  of  PDP-11  address  space  in  DEC-10 
pages  (512  words)  by  the  DEC-10  processor 

-  Software-selectable  write  protect  and  sign  extend  on  DEC-10 
pages  by  the  DEC-10  processor 

Software-selectable  mapping  of  1-1  or  2-1  PDP-11  words  per 
DEC-10  word  by  the  DEC-10  processor 

-  Bidirectional  and  programmable  interrupts 
Software-selection  of  interrupt  levels  by  the  DEC-10  processor 
for  PDP-11  interrupts  to  the  DEC-10 

-  Software-selection  of  7-bit  interrupt  vector  address  and  BR 
level  by  the  DEC-10  for  DEC-10  interrupts  to  the  PDP-11 

-  Two  software  addressable  16-bit  registers  for  communications 
between  the  DEC-10  and  each  PDP-11 

-  Remote  operation  of  PDP-11' s  of  up  to  150  ft  with  electrical 
isolation 

-  PDP-11  memory  access  to  MSM  without  DEC-10  CPU  control  for 
PDP-11  stand-alone  testing 

7.0  MSM  Prototype 

7 . 1  Description 

The  single  channel  MSM  (prototype)  for  AVSAIL  is  a  two  port,  single 
Memory,  Sequencer,  and  Interface  (MSI)  module  unit.  It  represents  the 
minimum  configuration  for  MSM.  The  prototype  has  one  UNIBUS  interface, 
one  DECsystem-10  Memory  Bus  interface,  and  one  DECsystem-10  I/O  Bus 
interface.  It  contains  16K  of  DECsystem-10  memory  and  an  operational 
front  panel.  The  MSM  resides  in  a  single  upright  cabinet  rack  on 
casters  with  one  card  cage  housing  eight  wire-wrap  cards.  Two  other 
wire-wrap  cards  are  present  within  the  system.  One  is  mounted  behind 
the  front  panel  and  the  other  resides  in  the  PDP-11  cabinet  on  the 
UNIBUS.  A  150  ft  cable  connects  the  PDP-11  to  the  MSM  cabinet.  A  front 
view  drawing  of  the  MSM  prototype  is  shown  in  Figure  7. 


FIGURE  7.  SINGLE  CHANNEL  MSM  (PROTOTYPE) 


7.2  Specifications 
General: 


MSM  Configuration: 
No.  of  Channels: 

No.  of  Ports: 
Scanner  Clock  rate: 


Single  MSI 
1 
2 

20  MHZ 


Memory: 


Size : 

Word  length: 

Type : 

Memory  Cycles: 

Read  Access  time*: 


16  K  words 

36  bits  +  4  parity  bits 
HMOS  RAM  (35  ns  access) 

KI10:  Read,  write,  read-modify-write 
PDP-11/40:  Read,  write 
KI10:  850  ns 

PDP- 11/40:  2.2  us  (current) 

1.6  us  (with  SSYNC  mod) 


Mechanical  &  Environmental: 

Cabinet  Size  (HxWxD) : 
Weight: 

Input  Power: 

Power  supplies: 

Operating  current: 

Operating  temperature: 
Relative  humidity: 


69  7/8"  x  22"  x  25" 

Less  than  500  lbs 

115  VAC  60  Hz  single  phase 

+5  VDC  ±10%  @  75A,  30°C 

-5  VDC  ±10%  @  10A,  30°C 

8  -  8  Amps 

10°C  to  50°C 

20%  to  95%,  non-condensing 


*time  is  figured  for  best  case  with  immediate  service  granted  to  port. 
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